Inspired from human cognition, machine learning systems are gradually revealing advantages of sparser and more modular architectures. Recent work demonstrates that not only do some modular architectures generalize well, but they also lead to better out of distribution generalization, scaling properties, learning speed, and interpretability. A key intuition behind the success of such systems is that the data generating system for most real-world settings is considered to consist of sparse modular connections, and endowing models with similar inductive biases will be helpful. However, the field has been lacking in a rigorous quantitative assessment of such systems because these real-world data distributions are complex and unknown. In this work, we provide a thorough assessment of common modular architectures, through the lens of simple and known modular data distributions. We highlight the benefits of modularity and sparsity and reveal insights on the challenges faced while optimizing modular systems. In doing so, we propose evaluation metrics that highlight the benefits of modularity, the regimes in which these benefits are substantial, as well as the sub-optimality of current end-to-end learned modular systems as opposed to their claimed potential.

Deep learning research has an established history of drawing inspiration from neuroscience and cognitive science.

Deep learning research has an established history of drawing inspiration from neuroscience and cognitive science.

Inspired from human cognition, machine learning systems are gradually revealing advantages of sparser and more modular architectures. Recent work demonstrates that not only do some modular architectures generalize well, but they also lead to better out of distribution generalization, scaling properties, learning speed, and interpretability. A key intuition behind the success of such systems is that the data generating system for most real-world settings is considered to consist of sparse modular connections, and endowing models with similar inductive biases will be helpful. However, the field has been lacking in a rigorous quantitative assessment of such systems because these real-world data distributions are complex and unknown. In this work, we provide a thorough assessment of common modular architectures, through the lens of simple and known modular data distributions.

Online corner case detection is crucial for ensuring safety in autonomous driving vehicles. Current autonomous driving approaches can be categorized into modular approaches and end-to-end approaches. To leverage the advantages of both, we propose a method for online corner case detection that integrates an end-to-end approach into a modular system. The modular system takes over the primary driving task and the end-to-end network runs in parallel as a secondary one, the disagreement between the systems is then used for corner case detection. We implement this method on a real vehicle and evaluate it qualitatively. Our results demonstrate that end-to-end networks, known for their superior situational awareness, as secondary driving systems, can effectively contribute to corner case detection. These findings suggest that such an approach holds potential for enhancing the safety of autonomous vehicles.

Underwater structures that can change their shapes dynamically, the way fish do, push through water much more efficiently than conventional rigid hulls. But constructing deformable devices that can change the curve of their body shapes while maintaining a smooth profile is a long and difficult process. MIT's RoboTuna, for example, was composed of about 3,000 different parts and took about two years to design and build. Now, researchers at MIT and their colleagues--including one from the original RoboTuna team--have come up with an innovative approach to building deformable underwater robots, using simple repeating substructures instead of unique components. The team has demonstrated the new system in two different example configurations, one like an eel and the other a wing-like hydrofoil.

Underwater structures that can change their shapes dynamically, the way fish do, push through water much more efficiently than conventional rigid hulls. But constructing deformable devices that can change the curve of their body shapes while maintaining a smooth profile is a long and difficult process. MIT's RoboTuna, for example, was composed of about 3,000 different parts and took about two years to design and build. Now, researchers at MIT and their colleagues -- including one from the original RoboTuna team -- have come up with an innovative approach to building deformable underwater robots, using simple repeating substructures instead of unique components. The team has demonstrated the new system in two different example configurations, one like an eel and the other a wing-like hydrofoil.

Deep neural networks (DNNs) have drawn much inspiration from the human cognitive process, evidenced recently in their incorporation of modular structures and attention mechanisms. By representing knowledge in a modular manner and selecting relevant information via attention mechanisms, DNN models can develop meaningful inductive biases, boost their out-of-distribution generalization abilities, and manipulate concepts at higher levels of cognition. While modular architectures provide proven advantages for DNNs, there currently exists no rigorous quantitative assessment method for them due to the complexity and unknown nature of real-world data distributions. As such, it is unclear whether or to what extent the performance gains obtained by modular systems are actually attributable to good modular architecture design. In the new paper Is a Modular Architecture Enough, a research team from Mila and the Université de Montréal conducts a rigorous and thorough quantitative assessment of common modular architectures that reveals the benefits of modularity and sparsity for DNNs and the sub-optimality of existing end-to-end learned modular systems.

Rockwell Automation, the world's largest company dedicated to industrial automation and digital transformation, announced that it has signed a definitive agreement to acquire CUBIC, a company that specializes in modular systems for the construction of electrical panels. CUBIC, founded in 1973, serves fast-growing industries, such as renewable energy, data centers, and infrastructure, and is headquartered in Bronderslev, Denmark. CUBIC's efficient and flexible modular systems combined with Rockwell's intelligent devices and industry expertise will benefit customers by offering faster time to market, enabling broader plant-wide applications for intelligent motor control, and generating smart data to increase sustainability and productivity. CUBIC's established partner model will allow Rockwell to build an expanded Partner Network for intelligent motor control offerings in Asia, Europe, and Latin America. The company will bring new customers and partners in hybrid and process industries.

Intelligent systems for the annotation of media content are increasingly being used for the automation of parts of social science research. In this domain the problem of integrating various Artificial Intelligence (AI) algorithms into a single intelligent system arises spontaneously. As part of our ongoing effort in automating media content analysis for the social sciences, we have built a modular system by combining multiple AI modules into a flexible framework in which they can cooperate in complex tasks. Our system combines data gathering, machine translation, topic classification, extraction and annotation of entities and social networks, as well as many other tasks that have been perfected over the past years of AI research. Over the last few years, it has allowed us to realise a series of scientific studies over a vast range of applications including comparative studies between news outlets and media content in different countries, modelling of user preferences, and monitoring public mood. The framework is flexible and allows the design and implementation of modular agents, where simple modules cooperate in the annotation of a large dataset without central coordination.